Skip to content

Apply schema transformers on properties and other subschemas #56709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 17, 2024

Conversation

captainsafia
Copy link
Member

@captainsafia captainsafia commented Jul 10, 2024

Closes #56584.

Schema transformers are currently only called on schemas that are generated at the top-level for respones, parameters, and request bodies. This PR allows schema transformers to be applied recursively into sub-schemas that might appear in these arguments (for example, the schema for an int property that is part of a Todo type). We currently apply transformations to subschema types where we understand how to resolve TypeInfo for them:

  • schema.items: for schemas that represent lists of elements
  • schema.anyOf: for schemas that represent polymorphic schemas with subtypes
  • schema.properties: for schemas associated with properties in a type

Changes in this PR include:

  • Remove the OpenApiSchemaTransformerContext.Type property and adding OpenApiSchemaTransformerContext.JsonTypeInfo and OpenApiSchemaTransformerContext.JsonPropertyInfo
  • Changes to OpenApiSchemaService to support applying schemas recursively into the subschemas mentioned above
  • Additional tests in SchemaTransformerTests to cover relevant scenarios

Note: One of the new tests introduced here (SchemaTransformer_CanModifyListOfPolymorphicTypes) is currently skipped because of a bug I discovered in ApplyPolymorphismOptions where we don't handle things correctly if a polymorphic type is a child of another type (like List<Shape> or a type with a Shape property) because of the context.Path.Length == 0 check we currently use to check for base types. This should be resolved once we have can consume dotnet/runtime#104046 in an upcoming runtime update.

Another note: I'm hoping to bring these changes in to #56599 for more complete perf coverage for these transformer types.

@captainsafia captainsafia requested a review from a team as a code owner July 10, 2024 00:34
@ghost ghost added the old-area-web-frameworks-do-not-use *DEPRECATED* This label is deprecated in favor of the area-mvc and area-minimal labels label Jul 10, 2024
@captainsafia captainsafia requested a review from mikekistler July 10, 2024 00:34
@captainsafia captainsafia added area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates feature-openapi area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc and removed old-area-web-frameworks-do-not-use *DEPRECATED* This label is deprecated in favor of the area-mvc and area-minimal labels labels Jul 10, 2024
Copy link
Contributor

@mikekistler mikekistler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comments/questions that I hope are helpful.

@captainsafia
Copy link
Member Author

I left a few comments/questions that I hope are helpful.

Posting a response to all the comments here since they all relate to the same concept: the constraint associated with this implementation that we only apply transformations to subschemas that we can resolve a JsonTypeInfo for. The subschemas that we don't recurse into are:

  • schema.not
  • schema.allOf
  • schema.oneOf
  • schema.additionalProperties

If a user sets one of these subschemas explicitly in their own transformer, we have no way of constructing a complete context object for it because we don't know the type associated with the schema.

options.UseSchemaTransformer((schema, context, cancellationToken) =>
{
	// What is the JsonTypInfo assocaited with this schema?
	schema.Not = new OpenApiSchema { Type = "string" };
});

There's two approaches to solving this:

We could relax the constraints on the OpenApiSchemaTransformerContext so that the JsonTypeInfo is optional and recurse into the subschemas above. At the point, there's no way for you to know what the context of the schema you're modifying is. Is it a not schema, is it an allOf? There's no way to tell without adding additional state to the context object.

Alternatively, we could set the OpenApiSchemaTransformerContext.JsonTypeInfo property to the parent type associated with the schema (for example, a not inside a Todo schema would have JsonTypeInfo = Todo. This has the opposite problem though: you'd recurse into a schema with the wrong type and potentially make unexpected modifications to the target schema.

So, stuck between a rock and a hard place, I opted not to recurse into these as part of the default implementation. If a user explicitly needs it though, it should be possible to construct their own schema transformer implementation that supports the recursion into these properties and they can initialize the OpenApiSchemaTransformerContext with whatever state makes sense for their scenario. I'll add a test to model how I think this approach would look.

Let me know if you have any other questions or if there is some other way to solve the ambigous context problem.

@captainsafia
Copy link
Member Author

Bump for reviews! Hoping to get this in before Friday...

@martincostello
Copy link
Member

FWIW I had a look at it last week and didn't spot anything.

Copy link
Member

@BrennanConroy BrennanConroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only worry is the implicit relationship between schema properties and jsonTypeInfo. Any sensible bounds checking we can add?

{
var derivedJsonTypeInfo = _jsonSerializerOptions.GetTypeInfo(derivedType.DerivedType);
context.UpdateJsonTypeInfo(derivedJsonTypeInfo, null);
await InnerApplySchemaTransformersAsync(schema.AnyOf[anyOfIndex], derivedJsonTypeInfo, context, transformer, cancellationToken);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

schema.AnyOf[anyOfIndex]

I don't see any bounds checking here. If it's done earlier, it's far from this callsite and doesn't feel safe.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this takes an assumption on the implementation details of the schema generator which generates the anyOf child schemas based on the ordering in JsonTypeInfo.DerivedTypes. I added a check here but things will get spooky for people if they are inserting subschemas into anyOf that don't match what is understood by STJ from the derived type attributes.

foreach (var propertyInfo in jsonTypeInfo.Properties)
{
context.UpdateJsonTypeInfo(_jsonSerializerOptions.GetTypeInfo(propertyInfo.PropertyType), propertyInfo);
await InnerApplySchemaTransformersAsync(schema.Properties[propertyInfo.Name], jsonTypeInfo, context, transformer, cancellationToken);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same kind of deal, it seems like there is an implicit very close relationship between schema and jsonTypeInfo?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I introduced a check here to only apply transformers on properties with matching names. This does mean that there is a gap for scenarios where a user applies a schema transformer that changes the casing/format of property keys in a way to is inconsistent with how System.Text.Json is configured. In that case, they'd have to fallback to applying the recursion manually themselves in a top-level schema transformer since this would no-op.

@captainsafia captainsafia force-pushed the safia/recursive-schema-transformers branch from c44268b to c525ccd Compare July 16, 2024 20:26
@captainsafia captainsafia merged commit 8b39e65 into main Jul 17, 2024
26 checks passed
@captainsafia captainsafia deleted the safia/recursive-schema-transformers branch July 17, 2024 18:42
@dotnet-policy-service dotnet-policy-service bot added this to the 9.0-preview7 milestone Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates feature-openapi
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose additional info about properties and JSON types in OpenApiSchemaTransformerContext
4 participants